This invention relates generally to the field of image recognition and processing and specifically to methods and systems for identifying, diagnosing, and treating people based on thermal minutiae within a person""s body, primarily the face.
Improved methods for automated access control and surveillance are vital to ensure the continued security of nuclear weapon storage facilities, as well as other sensitive or valuable items. Potential threats range from terrorist bombings, insider thefts, and industrial espionage to sabotage by environmental activists. There is concern for increased vigilance in the protection of critical strategic assets.
Current technology being used for access control is not sufficient reliable, secure, fast rugged or cost effective for routine unattended operations at high-security locations. The challenge is to develop systems to secure facilities and personnel from internal and external threats in a cost effective and timely manner. Replacing human guards with automated systems can provide a significant cost savings.
The requirement to positively identify each individual seeking access to a facility or to information or services is widespread. Manpower-intensive guard brigades are deployed at public functions to protect celebrities, and at locations where valuable or important items are stored. Guards are used to screen entrants based upon recognizing either the person or some credential he carries. Identification credentials such as photo ID badges and driver""s licenses are widely used for manual identification when cashing checks or using credit cards. Manual checking of such identification cards may not recognize cases where the card is forgery or where the person using it is not the rightful owner of the card. To assist in solving that problem, more sophisticated identifying characteristics may be used on the card, and features may be added to make the card more difficult to counterfeit. The use of biometric characteristics such as fingerprints, signatures, visual descriptions, or photographs is becoming more common. Such information can either be readable manually or encoded for reading by an automated system.
When the identification system is fully automated, without a human attendant, biometric sensors at the access location can compare the characteristics of a person at the location with the stored characteristics of the person he is claiming to be. When initially issuing permission for a person to access a biometrically-controlled system or location, his biometric characteristics are recorded in the system memory, and also recorded on an identification card, for later comparison by the system controller with his live characteristics.
Current biometric identification systems include use of inkless fingerprint systems (called xe2x80x9clive scanxe2x80x9d units), retinal scanners, hand geometry measuring devices, voice recognition, handwriting recognition, and facial recognition systems which use either visual or infrared cameras. Use of fingerprints is generally considered the most secure method for positive identification. However, when used in the unattended mode, fingerprints can be lifted from one location or surface and positioned at another location. Therefore unattended use of fingerprints for identification at locations requiring very high security is not acceptable. A more common identification to widespread use of fingerprints for identification is the requirement for placing one or more clean fingers on a glass plate for imaging by the fingerprint recognition system. This requires that the hands be free and relatively clean, and that the glass plate be maintained intact and clean. The plates are vulnerable to vandalism. When used for access control at a busy location, there is a time delay associated with unloading the hands and positioning the fingers properly. Also, users must cooperate with the system. In certain scenarios of use, cooperation of the subject may be difficult to obtain. Furthermore, many persons have a reluctance to being fingerprinted for an identity card, since they associate the process with criminal activities.
Fingerprints traditionally have been the sole means of positive identification admissible as evidence in criminal trials in the U.S. Fingerprinting of criminals, military personnel, persons seeking security clearances, and persons applying for sensitive jobs has been performed for many years. The FBI established and maintained a card file in which each persons""s fingerprints were printed by rolling the fingers first on an inked pad and then on the card. Much of the original FBI fingerprint file of rolled prints has now been digitized and made available on-line for computer access. The process of digitizing the historical files, and the continuing task of maintaining current fingerprint files, has cost hundreds of millions of dollars during the past ten years alone. Aside from the labor costs of performing the digitization and managing the search tasks through the database, significant RandD has been performed to develop specialized software for comparing unknown fingerprints against the database within a reasonable period of time, and specialized hardware has been developed to provide rapid response.
Inkless techniques are now generally used to produce a xe2x80x9ctenprintxe2x80x9d card which substitutes for the former rolled print card. Common inkless techniques utilize polarized light to illuminate the fingers, and light sensors to image the light reflected and refracted from the ridges. The resulting image can be more consistent and higher quality than the rolled prints, since inconsistencies in the amount of ink applied and in the pressure used to transfer the print to paper are not a factor.
Automated fingerprint matching techniques have been developed which rapidly classify an unknown print and then search through the portion of the database associated with that class looking for a match. Unknown prints may be from a xe2x80x9ctenprintxe2x80x9d card, or may be latent prints which have been lifted from a crime scene. A latent print may include a sizable area of one or more fingers, such as on a water glass, or it may include only a portion of one or more fingers, such as on a telephone keypad. Latent prints may be found on top of other latent prints, such as when several people been used the same telephone.
Matching techniques often extract minutiae points from the prints, and then compare the sets of minutiae rather than compare entire prints. Various classifications of minutiae types have been proposed by different companies and authorities. An example is given from the Costello U.S. Pat. No. 4,947,443. Six types of xe2x80x9ccharacteristic featuresxe2x80x9d are presented in this patent, each one relating to a type of minutia. This fingerprint matching technique references the type, orientation, and location of each characteristic and each and every other characteristic. Using this approach, on the order of 80 to 150 minutia points are identified in each fingerprint. Other fingerprint minutiae extraction and matching patents produce essentially the same number of minutiae, with difference in what features of the set of minutiae are considered in attempted matching and in how the matching is performed. In U.S. courts, evidentiary rules have traditionally required that 16 or more minutia points be found to correspond between two prints in order for them to be considered to be from the same person. The determination of likely matching prints is generally, assisted or performed entirely by a computer system; however, the final decree of a match is made by a fingerprint expert, who reviews the computer system results.
Matches between different prints taken from the same finger are never perfect, since the fingers are deformable, three-dimensional, connected and jointed structures which leave two-dimensional prints on surfaces they encounter through pressure. The exact angles between the fingers and the surfaces, the amount and direction of pressure, and the effect of movement between the fingers and the surfaces all cause variations in the exact prints produced. Even when prints are produced by a live scan technique, variation in the lighting, hand position, oil or dust on the fingers, use of lotions, and scratches or paper cuts will produce mirror variations in the prints produced.
Therefore, the exact number, position, and characteristics of minutiae extracted from two prints may be different even though they are produced by the same finger. The challenge for an automated fingerprint identification system is to recognize allowable mirror variations in actual matching prints while not allowing variations so wide that mismatches occur. Several AFIS products are now commercially offered which provide acceptable accuracy. Local and regional police forces may use smaller databases which contain only the prints of persons historically associated with their areas, rather than relying on federal resources to search the entire nationwide FBI files. Smaller scale fingerprint system, such as those associated with a system which controls access to an office building, may use the same minutiae matching techniques.
With rolled and live-scan prints, the orientation of each print, and the finger to which it corresponds is known. Also, quality checks can be built into the process such that repeat prints may be taken to insure quality when needed. In the case of latents, however, the analysis is done after the fact. It is known which finger left the print, and the orientation of the finger may be in doubt when only a partial print is found. Therefore, matching of latents is much more difficult than matching of rolled or live scan prints.
Various minutiae extraction algorithms are used in current fingerprint identification systems, some of which merely utilize the location of the minutia points and others of which utilize also additional information about the type of minutia each point represents. For example, simple graph matching techniques can be used to compare the follow-the-dots vectors generated by connecting the minutia points in order forced by considering intersections with a spiral from the centerpoint of the fingerprint. Alternately, the ridge angle at each minutia point can be considered and matched along with the coordinates, in a best-fit attempt to match each unknown print to each known print. A measure of goodness of fit can then be computed and used to rank other possible matches.
U.S. Pat. No. 4,525,859 to Bowless teaches a pattern recognition system which detects line bifurcations and line endings, denoted minutiae, in a pattern of lines such as are found in a fingerprint. According to this reference, the FBI uses an automatic fingerprint identification system entitled xe2x80x9cFINDERxe2x80x9d which uses an optical scan reader. The information is then enhanced to eliminate grays and fill in gaps in the ridges. A 16xc3x9716 increment square window scans the fingerprint, an increment being a tenth of a millimeter. Thus, a window advances through the fingerprint in increments of tenth of millimeter and looks for ridges which enter the window but do not exit it. When such a ridge is identified, its coordinate location is stored and the ridge is analyzed to establish an angle, theta, of the ridge at the termination. The data are then re-scanned to look for terminations of valleys, which are ridge bifurcations. The additional coordinates and angles of each of the inverted ending points also are stored.
In latent points, the distances between ridges of a fingerprint average 0.4 millimeters but can vary by a factor of 2 for any individual finger depending on skin displacement when the finger contacts the hard surface normally encountered in establishing a print.
A known algorithm of the National Institute for Standard Technology can be used to compare a previously stored electronic image of minutiae coordinate locations with the minutiae locations identified and stored by the computer.
U.S. Pat. No. 5,040,224 to Hara discloses a fingerprint processing system capable of detecting a core of a fingerprint image by statistically processing parameters. Hara""s invention provides a system to determine a core in the fingerprint image and/or to detect directions and curvatures of ridges of the fingerprint image prior to detection of the position of the core. This reference defines minutiae as abrupt endings, bifurcations, and branches.
U.S. Pat. No. 4,790,564 to Larcher teaches a process and apparatus for matching fingerprints based upon comparing the minutiae of each print in a database with precomputed vector images of search minutiae in a search print to be identified, comparing position and angle, a result of such comparison being a matching score indicating the probability of a match between the angle of a file print minutiae and the angle of precomputed vector images of the search minutiae. Over an under-inking of a rolled print can change the apparent type of minutiae associated with a particular point from one printing to the next. However, not all corresponding minutiae will appear to change type in the two pairs. Therefore, matching for type as well as for x and y coordinates provides a stricter match requirement and results in better system accuracy. Larcher assigns higher values to minutiae which match in x,y and type.
As Larcher points out, there are advantages to matching minutiae rather than the entire image of the fingerprint in itself. An elementary matching operation comprises the comparison of two sets of minutiae, i.e., two sets of points, each point having three coordinates x, y, and a. An elementary matcher attempts to superimpose the two sets of points, in order to count the number of minutiae which are common to the two fingerprints.
Numerous other schemes for matching fingerprints are known. For example, matches referred to in Wegstein, Technical Note 538 of the National Bureau of Standards (1970), as M19, M27, and M32, determine whether two fingerprints come from the same finger by computing the density of clusters of points in Dx-Dy space, where Dx and Dy are the respective differences in x and y coordinates for the minutiae of two fingerprints. Experimental results referred to in this reference indicate that in Dx-Dy space points tend to be located at random when coming from different fingerprints, whereas points tend to form a cluster when coming from fingerprints from the same finger.
In the M19 matcher, the assumption is made that the transformation needed to superimpose the two sets of minutiae points is a translation only. The M27 matcher is an M19 matcher with a new scoring function, intended to take into account greater translation displacements. The M32 matcher takes into account small rotations between two fingerprints in the following way: first an M27 matcher comparison is made between the two fingerprints; then, one of the two prints is rotated through xe2x80x9cVxe2x80x9d degrees from its original position and a new M27 comparison is made. All together an M32 matcher operation consists of seven M27 comparisons, corresponding to the following values for the angle V, i.e., V=xe2x88x9215, xe2x88x9210, xe2x88x925, 0, +5, +10, +15 degrees.
Minutiae may be extracted manually or automatically. Automatic systems generally require better quality imagery. The matcher engine must allow for some degree of inaccuracy or variability with respect to each of the encoded coordinates due to human operator bias or precision limitations of automated feature extraction processes.
Larcher disclosed the use and comparison of type of minutiae, since there is a greater match accuracy when ridge endings are compared to ridge endings, and bifurcations to bifurcations, as opposed to comparing one ridge ending to one bifurcation.
Other known approaches compare two sets of image features points to determine if they are from two similar objects as disclosed for example in Sclaroff and Pentland, MIT Media Laboratory, Perceptual Computing Technical Report #304. This reference suggests that first a body-centered coordinate frame be determined for each object, and then an attempt be made to match up the feature points.
Many methods of finding a body-centered frame have been suggested, including moment of inertia methods, symmetry finders, and polar Fourier descriptors. These methods generally suffer from three difficulties: sampling error, parameterization error, and non-uniqueness. The technique used in Sclaroff and Pentman disclosure has the limitation that it cannot reliably match largely occluded or partial objects.
Known techniques associated with fingerprint minutiae extraction and matching can be summarized as follows:
First, an unknown finger is scanned optically:
Second, the image is divided into pixels, where the size of the pixel relates to the quality of the result desired;
Third, certain pixels are selected as minutiae points;
Fourth, each minutia is assigned a vector having magnitude and directional information in relation to the surrounding characteristics of the fingerprint. Typically for each fingerprint, there would be a substantial number of minutia vectors characterizing its image;
Fifth, the set of minutia vectors of the unknown print are compared by computer to the set of vectors of known prints; and
Sixth, the comparison results are used to select potential matches and provide a goodness of fit indication between the unknown and known prints.
Numerous approaches to recognition using visible light imaging of faces have been proposed. Many of them apply standard pattern matching techniques, others involve definition of face metrics.
U.S. Pat. No. 4,975,969 to Tal discloses a method and apparatus for uniquely identifying individuals by measurement of particular physical characteristics viewable by the naked eye or by imaging in the visible spectrum. This reference defined facial parameters which are the distances between identifiable parameters on the human face, and/or ratios of the facial parameters, and teachers that they can be used to identify an individual since the set of parameters for each individual is unique.
Tal""s approach utilizes visible features on the face, and therefore cannot be relied upon to distinguish between faces having similar visual features, for example as would be the case with identical twins. In addition, the xe2x80x9crubber sheetingxe2x80x9d effect caused by changes in facial expression, the aging effects which cause lengthening of the nose, thinning of the lips, wrinkles, and deepening of the creases on the sides of the nose, all cause changes in the parameters and in ratios relied on in this method. Furthermore, the parameters and ratios of any particular person""s face may be measured by anyone taking a photograph, and thereby used to select or disguise another person to appear to be that person. Therefore, the security provided by such a technique may not be adequate for unattended or highly sensitive locations.
Still another known scheme utilizes eigenanalysis of visual face images to develop a set of characteristic features. Pentland, View-Based and Modular Eigenspaces for Face Recognition, MIT Media Laboratory Perceptual Computing Section, Technical Report No. 245. Faces are then described in terms of weighting on those features. The approach claims to accommodate head position changes and the wearing of glasses, as well as changes in facial expressions. This disclosure teaches that pre-processing for registration is essential to eigenvector recognition systems. The processing required to establish the eigenvector set is extensive, especially for large databases. Addition of new faces to the database requires the re-running of the eigenanalysis. Accordingly, use of eigenanalysis may not be appropriate for use in a general face identification system such as would be analogous to the FBI""s and AFIS fingerprint system.
Visible metrics typically require ground truth distance measurements unless they rely strictly upon ratios of measurements. Thus, such systems can be fooled by intentional disguises, and they are subject to variations caused by facial expressions, makeup, sunburns, shadows and similar unintentional disguises. Detecting the wearing of disguises and distinguishing between identical twins may be done from visible imagery if sufficient resolution and controlled lighting is available. However, that significantly increases the computational complexity of the identification task; and makes the recognition accuracy vulnerable to unintentional normal variations.
From the standpoint of evidentiary use, it might also be argued that the application of eigenanalysis to a very large database of faces, such as all mug shots in the FBI files, would be considered so esoteric by the public at large that automated matches based upon its use will not readily be acceptable to a jury as convincing evidence of identity. By comparison, techniques based on minutiae matching technique, such as are used with fingerprint identification, would be expected to find a more understanding reception by the law enforcement community, and to be more acceptable for evidentiary purposes within a reasonable number of years after their introduction.
One known scheme using facial thermograms for identification is described in the Prokoski et al U.S. Pat. No. 5,163,094 which discloses defining xe2x80x9celemental shapesxe2x80x9d in the surface thermal image produced by the underlying vascular structure of blood vessels beneath the skin. Depending on the environment of use, thermal facial identification may provide greater security over identification from visual images and may therefore be considered preferable. It is extremely difficult, if not impossible, to counterfeit or forge one face to look like another in infrared, whereas it is often possible to disguise one person to look like another in visible light. However, the use of elemental shapes is found in practice to be vulnerable to such variables as head rotation and tilt, ambient and physiological temperature changes, variations in imaging and processing systems, and distortions or obstructions in a facial image (e.g., due to eyeglasses).
Eigenanalysis of the elemental shapes of a thermal facial image has also been used for recognition. In one approach, several sets of elemental shapes are produced for each image by imposing different thermal banding constraints. The totality of shapes are then analyzed with respect to a library of facial thermal images. Eigenshape analysis is used to compare the characteristics of shapes in each person""s images. Eleven characteristics of each shape are considered, including: perimeter, area, centroid x and y locations, minimum and maximum chord length through the centroid, standard deviation of that length, minimum and maximum chord length between perimeter points, standard deviation of that length, and area/perimeter.
Each person""s image is then characterized by set of 11-coefficient vectors. The difference in eigenspace between any two images is calculated to yield a measurement to which a threshold was applied to make a xe2x80x9cmatch/no matchxe2x80x9d decision. In practice, such a system yields a useful method and apparatus for some applications. However, the calculation techniques for such a system are computationally intensive and require additional computational analysis of the entire database when new images are added. As with others of the prior known techniques, recognition is seriously impacted by edge effects due to head rotation and tilt, and by loss of definition in very cold or very hot faces.
None of the known techniques for facial analysis is believed to be sufficiently robust and computationally straightforward to allow practical application of such a scheme for highly sensitive unattended security applications.
Therefore, the need remains for a system and method that can be used to reliably recognize and verify the identity of an imaged person without manual assistance and without cooperation from the person being identified.
In accordance with the present invention, a system for recognizing faces comprises a thermal imaging device, a minutiae generator, a minutiae data generator, and a minutiae matcher. The thermal imaging device produces a signal representative of the thermal characteristics of a new face. The minutiae generator is connected to the thermal imaging device and produces a signal representative of thermal facial minutiae of the new face. The minutiae data generator stores minutiae data corresponding to known faces. The minutiae matcher is connected to the minutiae generator and the minutiae data generator and compares minutiae of the new face and of the known faces, producing a signal representative of a match between the new face and one of the old faces.
In another aspect of the invention, a method of recognizing faces senses thermal characteristics of known faces, identifies minutiae of the known faces, senses thermal characteristics of a new face, identifies minutia of the new face, determines a distance metric from each of the known faces to the new face, and determines a match between the new face and one of the old faces based on the distance metrics.
In still another aspect of the invention, faces are classified according to thermal minutiae, and facial minutiae data are encoded as a number of bits by overlaying a grid of cells on a thermal representation of face, setting a bit to a first state if any minutiae are located within the cell corresponding to that bit, and setting the bit to a second state if none of the minutiae are located within the cell corresponding to that bit.
In yet further aspects of the invention, other imaging modalities are used, and other body parts or objects are used, for minutiae-based recognition. Techniques for identifying medical patients, diagnosing medical conditions, identifying drug and alcohol users, and assisting with the positioning of surgical instruments are also achieved with the present invention.